Validity and Reliability of Scaffolded Peer Assessment of Writing From Instructor and Student Perspectives
نویسندگان
چکیده
Although peer reviewing of writing is a way to create more writing opportunities in college and university settings, the validity and reliability of peer-generated grades are a major concern. This study investigated the validity and reliability of peer-generated writing grades of 708 students across 16 different courses from 4 universities in a particular scaffolded reviewing context: Students were given guidance on peer assessment, used carefully constructed rubrics, and were provided clear incentives to take the assessment task seriously. Distinguishing between instructor and student perspectives of reliability and validity, the analyses suggest that the aggregate ratings of at least 4 peers on a piece of writing are both highly reliable and as valid as instructor ratings while (paradoxically) producing very low estimates of reliability and validity from the student perspective. The results suggest that instructor concerns about peer evaluation reliability and validity should not be a barrier to implementing peer evaluations, at least with appropriate scaffolds. Future research needs to investigate how to address student concerns about reliability and validity and to identify scaffolds that may ensure high levels of reliability and validity.
منابع مشابه
The Effect of Scaffolded Written Corrective Feedback on Iranian EFL Learners’ Writing Quality: An Activity Theory Perspective
With the latest paradigm shift in SLA from Cognitivism to Sociocultural Theory (SCT), more studies are carried out to investigate the efficacy of Written Corrective Feedback (WCF) through a social and cultural lens. A more recent offspring of SCT is Activity Theory which provides an explanatory framework for scrutinizing an activity in a social setting. The present study aimed at investigating ...
متن کاملPeering Inside Peer Review with Bayesian Models
Instructors and students would benefit more from computersupported peer review, if instructors received information on how well students have understood the conceptual issues underlying the writing assignment. Our aim is to provide instructors with an evaluation of both the students and the criteria that students used to assess each other. Here we develop and evaluate several hierarchical Bayes...
متن کاملSelf-, Peer-, and Teacher-Assessments in Writing Improvement: A Study of Complexity, Accuracy, and Fluency
Alternative assessment approaches and, among them, self-assessment and peer-assessment are becoming increasingly important in educational contexts. Designed to compare self-assessment, peer-assessment, and teacher-assessment, this study included 90 EFL students from 3 intact classes divided into 3 groups: self-assessment, peer-assessment, and teacher-assessment. After taking the TOEFL Proficien...
متن کاملThe Impact of Three Feedback Types on Postgraduate TEFL Student Teachers’ Writing Accuracy and Organization
One skill that student teachers need to develop during their academic studies is the capacity to produce accurate and well-organized texts. This study reports on the comparative impact of metalinguistic feedback (MLF), teacher interactive feedback (TIF(, and the peer-feedback (PF) on the accuracy and organization of postgraduate ELT student teachers’ writing. The participants were 57 postgradua...
متن کاملFacilitating Improvements in Laboratory Report Writing Skills with Less Grading: A Laboratory Report Peer-Review Process†
Incorporating peer-review steps in the laboratory report writing process provides benefits to students, but it also can create additional work for laboratory instructors. The laboratory report writing process described here allows the instructor to grade only one lab report for every two to four students, while giving the students the benefits of peer review and prompt feedback on their laborat...
متن کامل